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Abstract — We consider spatially coupled code ensembles. A 
particular instance are convolutional LDPC ensembles. It was 
recently shown that, for transmission over the binary erasure 
channel, this coupling increases the belief propagation thresh- 
old of the ensemble to the maximum a-priori threshold of 
the underlying component ensemble. We report on empirical 
evidence which suggest that the same phenomenon also occurs 
when transmission takes place over a general binary memoryless 
symmetric channel. This is confirmed both by simulations as 
well as by computing EBP GEXIT curves and by comparing the 
empirical BP thresholds of coupled ensembles to the empirically 
determined MAP thresholds of the underlying regular ensembles. 

We further consider ways of reducing the rate-loss incurred 
by such constructions. 



I. Introduction 

It has long been known that convolutional LDPC ensem- 
bles, introduced by Felstrom and Zigangirov [1], have excel- 
lent thresholds when transmitting over general binary-input 
symmetric-output memoryless (BMS) channels. The funda- 
mental reason underlying this good performance was recently 
discussed in detail in [2] for the case when transmission takes 
place over the binary erasure channel (BEC). 

In particular, it was shown in [2] that the BP threshold of 
the spatially coupled ensemble is essentially equal to the MAP 
threshold of the underlying component ensemble. It was also 
shown that for long chains the MAP performance of the chain 
cannot be substantially larger than the MAP threshold of the 
component ensemble. In this sense, the BP threshold of the 
chain is increased to its maximal possible value. This is the 
reason why we call this phenomena threshold saturation via 
spatial coupling. In a recent paper [3], Lentmaier and Fettweis 
independently formulated the same statement as conjecture. 
They attribute the observation of the equality of the two 
thresholds to G. Liva. 

It is tempting to conjecture that the same phenomenon 
occurs for transmission over general BMS channels. We pro- 
vide some empirical evidence that this is indeed the case. In 
particular, we compute EBP GEXIT curves for transmission 
over the binary additive white Gaussian noise (BAWGN) 
channel. We show that these curves behave in an identical 
fashion to the ones when transmission takes place over the 
BEC. We also compute fixed points (FPs) of the spatial 



configuration and we demonstrate again empirically that these 
FPs have properties identical to the ones in the BEC case. 

For a review on the literature on convolutional LDPC 
ensembles we refer the reader to [2] and the references therein. 
As discussed in [2], there are many basic variants of coupled 
ensembles. For the sake of convenience of the reader, we 
quickly review the ensemble (l, r, L, w). This is the ensemble 
we use throughout the paper. 

We assume that the variable nodes are at positions [— L, L], 
L £ N. At each position there are M variable nodes, M £ 
N. Conceptually we think of the check nodes to be located 
at all integer positions from [—00,00]. Only some of these 
positions actually interact with the variable nodes. At each 
position there are -M check nodes. It remains to describe 
how the connections are chosen. We assume that each of the 
1 connections of a variable node at position i is uniformly and 
independently chosen from the range [i, . , . , i + w — 1], where 
w is a "smoothing" parameter. In the same way, we assume 
that each of the r connections of a check node at position i 
is independently chosen from the range [i — w + 1, . . . , i]. 

A discussion on the above ensemble and a proof of the 
following lemma can be found in [2]. 

Lemma 1 (Design Rate): The design rate of the ensemble 
(l, r, L, w), with w < 2L, is given by 

l-2Eto(^) r 



1 lw 

R(l,r,L,w) = (l--) 

r r 



2L + 1 



II. Review: EBP GEXIT Curves, the Area Theorem, 
and the Maxwell Construction 

Our aim is to empirically demonstrate that the performance 
of coupled ensembles is closely related to that of the un- 
derlying ensemble also in the general case. We limit our 
discussion to the coupling of regular ensembles. To get started, 
let us briefly review how the BP and MAP threshold can be 
characterized for regular ensembles. A detailed discussion can 
be found in [4]. 

For £ > 1, the (forward) density evolution (DE) equation 
for an (l, r)-regular ensemble is given by 

x^c©^^)®^- 

Here, c is the i-density of the BMS channel over which 
transmission takes place and is the density emitted by 
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variable nodes in the £-th round of density evolution. Initially 
we have xo = Ao, the delta function at 0. The operators © 
and S correspond to the convolution of densities at variable 
and check nodes, respectively. We say that a density x is a 
FP of DE for the channel c if x = c © ( x E(*-i))©(i-i). More 
succinctly, we say that (c, x) is a FP of DE. 

For the BEC it is known that the behavior of the BP as 
well as the behavior of the MAP decoder are determined by 
the collection of FPs of DE. The same is conjectured to be true 
for general channels. Let us discuss this general conjecture. 

The key concept in this conjecture is the EBP GEXIT curve 
[5]. This curve is shown in Figure Q] for the (3, 6) -regular 
ensemble assuming that transmission takes place over the 
BAWGN. Numerically it is constructed in the following way. 
To construct one point on this curve find a FP (c CT ,x), where c CT 
denotes an element of the channel family under consideration. 
E.g., in the case considered in Figure Q] c CT represents the L- 
density of a BAWGN channel of variance a 2 . This FP gives 
rise to the point (H(c a ), G(c a , x)) in the GEXIT plot. Hereby, 



H(a) = J a(y)log(l + e- 
G(c CT ,a) = / a{y)l(a,y)dy 



J )dy, 



-dz)l 



l + e z +y '")' \J 1 

In words, H(-) computes the entropy associated to an L- 
density, whereas G(c a , ■) computes the so-called GEXIT value 
of an L-density. This GEXIT value depends on the "operating 
point", i.e., it depends on the underlying channel c a - To first 
order, the GEXIT value is equal to the entropy. 

We get the EBP GEXIT curve if we plot the points corre- 
sponding to all FPs of DE. For a detailed discussion we refer 
the reader to [4], [5]. It was shown in [5] that, under suitable 





Fig. 1. The EBP GEXIT curve for the (l = 3, r = 6)-regular ensemble 
and transmission over the BAWGNC. Each point on the curve corresponds 
to a FP (c ff , x) of DE. The two figures show the FP density x as well as the 
input density c CT for two points on the curves (see inlets). 

technical conditions, for every GEXIT value g G [0,1] there 
exists at least one FP (c CT ,x) with GEXIT value g. Further, 
a simple recursive numerical procedure can be used to find 
such a FP. The technical difficulty lies in establishing the 
existence of the curve (rather than the existence of just the set 
of FPs). Although the numerical evidence strongly suggests 
the existence of the curve, it is an open problem to prove this 
analytically. 



We can construct an upper bound on the MAP threshold as 
shown in Figure |2] see [5]: integrate the EBP GEXIT curve 
starting from the point (1,1) from right to left until the area 
under the curve equals the rate of the code. The point on the 
x-axis where this equality occurs is an upper bound on the 
MAP threshold. It is conjectured to be in fact the exact MAP 
threshold. 
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Fig. 2. Upper bound on the MAP threshold for the (3, 6)-regular ensemble 
and transmission over the BAWGNC. This upper bound is given by the entropy 
value where the dark gray vertical line hits the x-axis. The Maxwell conjecture 
states that this bound is tight. Numerically the upper bound is at a channel 
entropy of roughly 0.4792. For comparison, the BP threshold is at a channel 
entropy of roughly 0.4291. 



III. Density Evolution, Fixed Points, and the EBP 
GEXIT Curve for Coupled Ensembles 

A. Density Evolution 

Let us describe the DE equations for the (l, r, L, w) ensem- 
ble. In the sequel, densities are L-densities. Let c denote the 
channel density and let x, denote the density which is emitted 
by variable nodes at position i. 

Definition 2 (Density Evolution of (l, r, L, w) Ensemble): 
Let Xj, i G Z, denote the average L-density which is emitted 
by variable nodes at position i. For i G" [-L, L] we set 
x s ; = A +oc . Here, A +oc is the delta function at +oo. In 
words, the boundary variable nodes have perfect information. 
For i G [— L, L], the FP condition implied by DE is 



W — 1 -, w — 1 

( 1 S^f 1 sHfr-l)^ 1 - 1 ) 
I _ Z^V Zv X i+j-k) 



Define 



3=0 fc=0 



_^ w— 1 w— 1 



(1) 



w z — ' w z — ' / 

j=0 k=0 



k=0 

Note that g(x,...,x) = (x 18 ^ -1 ))®^ -1 ), where the r-h-s 
represents DE (without the effect of the channel) for the 
underlying (l, r)-regular ensemble. 

We write y -< x if x is degraded w.r.t. y. It is not hard to 
see [4] that the function g(x.i- w+ i, . . . , x^) is monotone w.r.t. 
degradation in all its arguments Xj, j = i — w + 1, . . . , i. 
More precisely, if we degrade any of the densities Xj, j = 
i — w + 1, . . . , i, then g(-) is also degraded w.r.t. to its original 
value. We say that g(-) is monotone in its arguments. ■ 
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B. Fixed Points and Admissible Schedules 

Definition 3 (FPs of Density Evolution): Consider DE for 
the (l,r, L, w) ensemble. Let x = (x_£, . . . , xl). We call 
x the constellation (of symmetric L-densities). We say that 
x forms a FP of DE with channel c if x fulfills (Q]) for 
i e [— L,L]. As a short hand we then say that (c,x) is a FP. 
We say that (c, x) is a non-trivial FP if x is not identically 
equal to A +oc Vi. Again, for i ^ [— L,L], >q = A +oc . ■ 

Definition 4 (Forward DE and Admissible Schedules): 
Consider forward DE for the (l,r, L, w) ensemble. More 
precisely, pick a channel c a . Initialize x' - 1 = (Ao, . . . , Ao). 
Let x}^ be the result of I rounds of DE. More precisely, 
x^ +1 ) is generated from xS^ by applying the DE equation 
(fl~|l to each section i 6 [-L, L], 



w + 1 , ■ ■ ■ ,A i _|_ 1(J _ 1( | 



We call this the parallel schedule. 

More generally, consider a schedule in which in each step 
I an arbitrary subset of the sections is updated, constrained 
only by the fact that every section is updated in infinitely 
many steps. We call such a schedule admissible. Again, we 
call xj e ) the resulting sequence of constellations. 

Lemma 5 (FPs of Forward DE): Consider forward DE for 
the (l, r, L, w) ensemble. Let xj^ denote the sequence of 
constellations under an admissible schedule. Then x^ con- 
verges to a FP of DE, with each component being a symmetric 
i-density and this FP is independent of the schedule. In 
particular, it is equal to the FP of the parallel schedule. 

Proof: Consider first the parallel schedule. We claim that 
the vectors xj^ are ordered, i.e., xj ^ >- x^ >~ ■ ■ ■ >~ (the 
ordering is section-wise and is the vector of A+oo). This is 
true since x} ' = (Ao, . . . , Ao), whereas x} 1 ' ~< (c, . . . , c) -< 
(Ao,...,Ao) = xj°\ It now follows by induction on the 
number of iterations and the monotonicity of the function g(-), 
Vi, that the sequence x^' is monotonically decreasing. More 
precisely, we have xjf +1 ' -< xf \ Hence, from Lemma 4.75 
in [4], we conclude that each section converges to a limit 
density which is also symmetric. Call the limit x°°. Since the 
DE equations are continuous it follows that x} 00 ' is a FP of 
DE (Q]i with parameter c. We call xj 00 ^ the forward FP of DE. 

That the limit (exists in general and that it) does not depend 
on the schedule follows by standard arguments and we will 
be brief. The idea is that for any two admissible schedules the 
corresponding computation trees are nested. This means that 
if we look at the computation graph of schedule lets say 1 
at time I then there exists a time £' so that the computation 
graph under schedule 2 is a superset of the first computation 
graph. To be able to come to this conclusion we have crucially 
used the fact that for an admissible schedule every section 
is updated infinitely often. This shows that the performance 
under schedule 2 is at least as good as the performance under 
schedule 1. Since the roles of the schedules are symmetric, 
the claim follows. ■ 

C. The EBP GEXIT Curve for Coupled Ensembles 

We come now to the key point, the computation of the 
EBP GEXIT curve. From previous sections we have seen that 



FPs of forward DE are well defined and can be computed 
by applying a parallel schedule. This procedure allows us to 
compute stable FPs. As discussed in Section [HI it was shown 
in [5] how to compute unstable FPs for uncoupled ensembles 
by a modified DE procedure in which the entropy is kept fixed 
and the channel parameter is varied. The same procedure can 
be applied for coupled ensembles. Figure [3] shows the result 




Fig. 3. EBP GEXIT curves of the ensemble (l = 3, r = 6, L) for L = 
1, 2, 4, 8, 16, and 32 and transmission over the BAWGNC. The BP thresholds 
are e BP (3, 6, 1) = 0.66324, e BP (3, 6, 2) = 0.54701, e BP (3, 6, 4) = 0.49031, 
e BP (3,6,8) = 0.47928, e BP (3,6,16) = 0.47918, e BP (3,6,32) = 0.47917, 
The light/dark gray areas mark the interior of the BP/MAP GEXIT function 
of the underlying (3, 6)-regular ensemble, respectively. 

of this numerical computation when transmission takes place 
over the BAWGNC. Note that the resulting curves look very 
similar to the curves when transmission takes place over the 
BEC, see [2]. For very small values of L the curves are far 
to the right due to the significant rate loss that is incurred at 
the boundary. For L around 10 and above, the BP threshold 
of each ensemble is very close to the MAP threshold of 
the underlying (3, 6)-regular ensemble, namely 0.4792. The 
picture strongly suggests that the same threshold saturation 
effect occurs for general channels as it was shown analytically 
to hold for the BEC in [5]. 

IV. A Possible Proof Approach 

So far the current discussion was empirical. Let us now 
quickly review which parts of the proof in [2] can be extended 
easily and which currently seem difficult. 

(i) Constellation parameter: For the BEC, entropy is equal 
to the Bhattacharyya parameter which is equal to the era- 
sure probability and which is also equal to twice the error 
probability. So, in this case any of those values is a natural 
quantity to parametrize constellations. For general channels, 
these parameters differ, and their choice is not necessarily 
equivalent for the purpose of the proof. 

(ii) Existence of FP: Although we did not explicitly state 
it in this short paper, the Existence Theorem 29 of [2], 
which guarantees the existence of a special FP of DE (c.f. 
Figure can be extended to the general case by considering 
the Battacharyya functional. The main technical difficulty in 
the proof arises due to the fact that we are now operating on 
a space of symmetric probability densities. So to extend the 
proof of the BEC, we need to define appropriate metrics in this 
space so that we can apply the necessary fixed-point theorems. 
Together with the use of extremes of information combining 
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methods, see [4], the proof extends. 

(iii) Shape of the constellation and the transition length: A key 
ingredient in the proof for the BEC was to show that any FP of 
DE has a very particular "shape." More precisely, any FP had a 
very "fast" transition between its extreme values. Empirically, 
for the general case we observe the same phenomena. From 
Figure [4] we see that the FP quickly saturates to its maximum 
value (w.r.t. physical degradation) of the stable FP of the 
(1, r)-regular ensemble. To show this property analytically 
seems currently to be one of the key difficulties in extending 
the proof. 

(iv) Construction of GEXIT Curve and the Area Theorem: 
Another key part of the BEC proof is the construction of a 
family of FPs (not necessarily stable FPs). The GEXIT curve 
plus the fast transition makes it possible in the case of the 
BEC to show that the "special" FP which was constructed via 
the existence theorem, has an associated channel parameter 
very close to the MAP threshold. How to best construct the 
GEXIT curve is an open issue. 
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Fig. 4. Unimodal (special) FP of the (l = 3, r = 6, L = 16, w = 3) 
ensemble for BAWGNC(cr) with a = 0.9480 (channel entropy M 0.4792). 
The bottom figure plots the entropy of the density at each section. The values 
are small at the boundary and essentially constant in the middle. The top 
figure shows the actual densities at sections 0, -4, -8, -12, -16. Notice that the 
mass shifts towards the right as sections go to -16. Also plotted in the top 
figure at section is the stable FP of DE for the (3, 6)-regular ensemble at 
<t = 0.9480. The density is right on the top of the one at section 0. 



V. How to Mitigate the Rate Loss 

We have seen that by coupling ensembles we can increase 
the threshold substantially. We also know that, due to the 
boundary condition, we incur a rate loss (see Lemma [TJ. This 
rate loss decreases linearly in 2L + 1, the length of the chain. 
Therefore, by picking L large we can ensure that the rate loss 
is as small as desired. But large L implies large codeword 
lengths and also may require a large number of iterations in 
the decoding process. This motivates to find ways of mitigating 
this rate loss. 

To keep things simple, we consider transmission over the 
BEC. The same techniques and trade-offs apply to general 
BMS channels, but of course the given numerical values will 
change. We discuss two basic techniques: (i) rather than setting 
all boundary variables to be known, it suffices to set to known 
only a smaller fraction; (ii) it suffices to start the process at 
only one boundary rather than both. In addition there might 
be a benefit to consider ensembles defined on a circle rather 



than a line. This symmetrizes all positions of the ensemble, 
which in turn might lead to more efficient implementations. 

A. Circular Ensembles 

Consider a "circular" ensemble. This ensemble is defined in 
a similar manner as the (1, r, L, w) ensemble except that the 
positions are now from to K — 1 and the index arithmetic 
is performed modulo K. This circular ensemble has design 
rate equal to 1 — l/r. If we let K = 2L + w and if we 
set w — 1 consecutive variable positions to then we recover 
the ensemble (l, r, L, w). This in itself gives a possibly more 
efficient way of implementing coupled ensembles. In this 
implementation all positions are symmetric, except for the 
received values, which are modified for the chosen w — 1 
consecutive positions. 

Let us now generalize the construction. For k <E [0, A' — 1] 
let Kk G [0, 1] denote the fraction of variable nodes at position 
k which we set to be known. E.g., if we set ko = «i = • • • = 
k w _2 = 1 and all other values to then we recover the 
previous case. Define k = As we will see shortly, from a 
rate perspective, it is not necessarily the best to set all variables 
at a certain position to 0. Further, it can be better to choose 
the "boundary" positions to be non-consecutive. This is why 
it is useful to introduce the above general model. 

To start, let us compute the design rate for the above set-up. 
We denote this ensemble by (1, r, K, w, k). 

Lemma 6 (Rate): The design rate R(l, r, K, w, k) of the 
ensemble (l,r, K, w,k) is given by 

1 _ i _ i S^Tr/fcfc - EJ=d i^k-jf] 

r r E^a-^) , 

Proof: The design rate is equal to 1 — C/V, where C 
and V are the number check and variable nodes which are not 
fixed a priori to 0. Let us call those variable nodes "free." 

Let us start with V. There are M variable nodes per section. 
A fraction Kj of those is permanently fixed to 0. Therefore, 
the number of free variable nodes is V = M Ylk=o(^ ~~ K <0- 

At each section there are M - check nodes. A check node 

r 

imposes a constraint on the free variable nodes if at least one 
of its connections goes to a free variable node. The probability 
that a particular edge of a check node at position k is connected 
to a frozen bit is equal to ^ YlJ^o K k-ji where all index 
arithmetic is modulo K. This implies that the probability that 
all r edges of a check node at position k are connected to 
frozen variables is equal to Y^j=o K k-jY- Therefore, C = 

MK\{1 - i EkJii E^o «* 7 i) r ). ■ 

Example 7 ( Contiguous and Uniform Boundary): 
Consider the (l = 3,r = 6, K, w,k) ensemble where 
we set Ko = Ki = ■ ■ ■ = Kiij-2 = k. We set all other values 
Kk equal to 0. For the choice k = 1 we know that we can 
achieve a threshold of roughly 0.48815 irrespective of the 
length of K. What happens if we pick k strictly less than 1? 

Let 8 denote the "effective" erasure probability at the bound- 
ary. I.e., 5 denotes the fraction of variables in the boundary 
which are free and which were erased during the transmission 
process. We have 5 = (1 — n)e. What is the threshold e BP that 
can be achieved (for arbitrary large K) for a given value of 
51 
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Figure [5] shows the plot of e BP according to DE for w = 
3, 4, 8, 16 and 32 as a function of S. For e.g. w = 3, 
up to 5 = 0.23 the achievable threshold is still equal to 
its maximal value, namely 0.48815. For higher values of 8 
the threshold gracefully decreases. For S = 0.23 we have 
K = 1 - 0.23/0.48815 = 0.529. For this value of k the 
corresponding rate for K = 25 is 0.478. This is considerably 
larger than 0.4604, which is the rate if n = 1. Indeed, the rate 
loss has almost been halved. 
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Fig. 5. Behavior of the threshold e BP for a uniform and consecutive boundary 
condition as a function of the effective erasure probability 8 at the boundary. 
The parameters are w = 3, 4, 8, 16 and 32. For io = 3,4 we choose K = 200 
and for for w — 8, 16, 32 we chose K = 400. The x-axis shows 8 = 
(1 — K)e Bp . The y-axis shows the achievable threshold e Bp . The curve for 
ui = 3 is the left-most curve and the curve for w = 32 is the right-most 
curve. We have <e bp rj 0.48815 up to <5 ss 0.23 for w = 3, up to 8 « 0.267 
for w = 4, up to 8 ss 0.3 for w = 8, up to <5 ss 0.31 for w = 16, and up 
to 8 « 0.32 for w = 32. For larger values of 8, e BP gracefully decreases to 
e BP ^ 0.4294. 



Example 8 (Non-Contiguous and Non-Uniform Boundary): 
We can do slightly better. Pick 5 = 0.22 and 5 2 = 0.30. 
Note that these two positions are non-contiguous. For this 
choice we still get a threshold of 0.48815. The corresponding 
rate for K = 25 is 0.4806, which is slightly higher than in 
the previous case. 

B. One-Sided Ensemble 

An alternative scheme is to define the ensemble on a line but 
to employ different terminations at the two boundaries. To be 
precise. Let the variable nodes be positioned from to K — 1. 
Assume the usual (l, r,w) case. At the "right" boundary use 
the following termination scheme. At position K there are 
^lw^i check nodes (instead of the usual M-). Any edge, 
which under the standard connection rules should connect to 
a check node at a position K or larger is mapped to a check 
node socket at position K. In expectation, exactly Ml 1 ^ 
such sockets are needed so that all check nodes at position K 
have degree exactly r. Therefore, locally the right-hand side 
boundary behaves exactly like a (1, r) ensemble and there is 
no rate loss associated with this boundary. 

On the left-hand side we proceed as in the standard ensem- 
ble. This reduces the rate-loss by a factor 2 compared to the 
standard ensemble. E.g., for K = 25 we get for our usual 
(3, 6)-ensemble a rate of 0.48. But we can do better. 

Example 9 (One-Sided Termination of (3,6) Ensemble): 
Let w = 3 and consider an ensemble in which the right 
boundary is terminated without rate loss as described above. 



Under the standard scheme, the check nodes at position 
have degree 2 and check nodes at position 1 have degree 4. 

Take a fraction a of the check nodes at position and merge 
them with check nodes at position 1, Those merged nodes at 
position 1 have degree 6 as in the regular case. As long as 
a is sufficiently small the threshold still remains unchanged. 
But this merging further reduces the incurred rate loss. 

A small note of caution might be in order at this place. Even 
though we can, as we just showed, mitigate the rate loss, this 
comes at some price - the number of required iterations will 
go up. A characterization of the involved trade off would be 
of high practical interest. 

VI. Conclusion 

Starting with the work by Felstrom and Zigangirov [1], it 
has been known that coupled ensembles have an excellent 
performance. This was confirmed via threshold computations 
by Sridharan, Lentmaier, Costello and Zigangirov for the BEC 
[6] and by Lentmaier, Sridharan, Zigangirov and Costello for 
general channels [7]. In [2] it was shown that for transmission 
over the BEC the BP behavior of coupled ensembles is 
essentially equal to the MAP behavior of the underlying 
ensemble. 

The current paper provides numerical evidence which sug- 
gest that exactly the same behavior occurs also for transmis- 
sion over general BMS channels. We have further extended 
some of the basic techniques and statements which were used 
in [2] to accomplish the proof for the BEC to the general setup. 
A complete proof is unfortunately still open. As discussed 
already in [2], such a proof would automatically show that it 
is possible to achieve capacity under iterative coding, and in 
addition, the convergence to capacity would be uniform over 
the whole class of BMS channels. 
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